Delivering the National Diabetes Prevention Program: Assessment of Outcomes in In-Person and Virtual Organizations

The Centers for Disease Control and Prevention's Diabetes Prevention Recognition Program (DPRP) has helped organizations deliver the National Diabetes Prevention Program (National DPP) lifestyle change program for over 10 years. Four delivery modes are now approved: in person, online (self-paced, asynchronous delivery), distance learning (remote, synchronous delivery), and combination (hybrid delivery using more than one delivery mode). We assessed outcomes using data from 333,715 participants who started the 12-month program between January 1, 2012, and December 31, 2018. The average number of sessions attended was highest for in-person participants (15.0), followed by online (12.9), distance learning (12.2), and combination (10.7). The average number of weeks in the program was highest for in-person participants (28.1), followed by distance learning (20.1), online (18.7), and combination (18.6). The average difference between the first and last reported weekly physical activity minutes reflected an increase for in person (42.0), distance learning (27.1), and combination (15.0), but a decrease for online (-19.8). Among participants retained through session 6 or longer, average weekly physical activity minutes exceeded the program goal of 150 for all delivery modes. Average weight loss (percent of body weight) was greater for in person (4.4%) and distance learning (4.7%) than for online (2.6%) or combination (2.9%). Average participant weight loss increased gradually by session for all delivery modes; among participants who remained in the program for 22 sessions, average weight loss exceeded the program goal of 5% for all delivery modes. In summary, if participants stay in the program, most have positive program outcomes regardless of delivery mode; they have some outcome improvement even if they leave early; and their outcomes improve more the longer they stay. This highlights the benefits of better retention and increased enrollment in the National DPP lifestyle change programs, as well as enhancements to online delivery.


Introduction
In 2010, the U.S. Congress authorized the Centers for Disease Control and Prevention (CDC) to create and lead the National Diabetes Prevention Program (National DPP).The National DPP is a partnership of public and private organizations collaborating to build a nationwide delivery system for a lifestyle change program (LCP) proven to prevent or delay the onset of type 2 diabetes in adults at high risk or those with prediabetes.The LCP is an evidencebased behavioral change intervention delivered by trained lifestyle coaches using a 12-month curriculum.Program quality assurance is overseen by CDC's Diabetes Prevention Recognition Program (DPRP), now entering its eleventh year of approving recognition for organizations delivering the National DPP LCP [1].The DPRP requires that organizations offer a minimum of 22 sessions over a 12-month period, with sessions covering CDC-approved curriculum modules.The DPRP maintains a database of participant data submitted by delivery organizations, which allows for the calculation of specific performance metrics, such as percent weight loss and average weekly physical activity minutes, thus providing information on organizations' effectiveness when delivering the National DPP LCP.
In 2012, the first organization was recognized by the DPRP for in-person curriculum delivery.Since then, the DPRP has implemented three sets of revisions to its standards and operating procedures (DPRP Standards) [2].Although each version of the DPRP Standards has adjusted the requirements for achieving various levels of CDC recognition, the collection of participant body weight and weekly minutes of physical activity has been constant throughout all versions.In 2015, the DPRP Standards approved virtual delivery of the LCP as an alternative to in-person delivery, with options for online or other (no specifications were given at the time for these options other than "virtually or via one or more distance-learning modalities").In 2018, the DPRP Standards further categorized other delivery into distance learning or combination.According to the DPRP Standards, online delivery is defined as participants logging into self-paced program sessions via a computer, tablet, or smartphone (i.e., asynchronous) with required live lifestyle coach interaction offered to each participant no less than once weekly during the first 6 months and once monthly during the second 6 months.Distance learning delivery is defined as the lifestyle coach provid-ing live delivery (i.e., synchronous) of session content in one location and participants calling in or videoconferencing from another location.Combination delivery is defined as delivery that uses a combination of any of the previously defined delivery modes for each individual participant by trained lifestyle coaches.Because this definition implies that at least one component of combination delivery must be synchronous, we can assume that most lifestyle coach interactions in these organizations are synchronous.
The objective of this paper is to describe the outcomes of retention (number of program sessions attended, number of weeks spent in the program, and number of days spent in the program), weight loss achieved during the program, and average weekly physical activity minutes reported by participants who had the opportunity to complete the 12-month LCP, with a specific focus on delivery mode.Analyzing these outcomes can help us understand program performance, identify what is working well, and highlight opportunities for improvement.3 Journal of Diabetes Research number of participants with sex "not reported" accounted for ~0.1% of the total.We placed age into one of three categories: 18-44, 45-64, or 65+ years.The DPRP requires that delivery organizations ask participants to identify their ethnicity as Hispanic or Latino or not Hispanic or Latino.Participants are also asked if they identify as one or more of the following races: American Indian or Alaska Native, Asian or Asian American, Black or African American, Native Hawaiian or other Pacific Islander, or White.For this analysis, we combined ethnicity and race responses into mutually exclusive groups, first categorizing by ethnicity and then further categorizing not Hispanic or Latino by race.Participants who identify as more than one race were categorized as multiracial, and those who did not report ethnicity or race were reported as such.We calculated each participant's initial BMI using the participant's height and the first reported body weight.We then categorized these as 25 (or 23 if Asian or Asian American)-29 kg/m 2 , or ≥ 30 kg/m 2 , indicating that the person had overweight or had obesity, respectively.
In this analysis, we used the following session-level information: the date of each session, body weight reported on the session date, and total number of physical activity minutes recorded for the week prior to the session date.

Data Analysis.
We examined retention in the lifestyle change program by three different metrics: (1) the number of sessions attended, (2) the number of weeks a participant spent in the program, and (3) the number of days a participant spent in the program, as well as how many days it took participants to reach each session number.To increase comparability across organizations, our retention analyses presented the number of sessions attended through 22, which is the required minimum.We know from data submitted that there are organizations offering more than the minimum of 22, but this number varies.We also capped the number of weeks attended at 44 due to organizations' variability in duration of delivery during the last 2 months of the program.Participants who spent more than 44 weeks in the program were included in this number.When assessing retention, the analysis was not restricted to participants who attended all 22 sessions and 44 weeks.In analyses of weight loss, for each session, we calculated participant weight loss as the difference between the weight at that session and their first recorded weight and then divided by the starting weight to get the percent weight loss.In analyses of physical activity, we calculated average weekly physical activity minutes for each participant by summing minutes for all sessions where valid minutes were recorded and then dividing by the number of those sessions.We also report the initial amount of physical activity reported, as well as the final amount reported.
All analyses were descriptive (means and medians), stratified by delivery mode, and conducted using SAS Enterprise 7.1 (SAS Institute Inc., Cary, NC).

Enrollment and Retention.
Enrollment demographics for participants included in the retention analysis are listed in Table 1 and were consistent with what we previously described in detail [3].This previous analysis showed that a higher proportion of men and participants aged 18-44 years enrolled in online and distance learning delivery programs than in in-person delivery programs.Retention measured by average number of sessions attended was highest for in-person participants (15.0), followed by online (12.9), distance learning (12.2), and combination (10.7) (Table 1).Of the four delivery modes, in-person delivery had the highest retention after attending one session (95.3%), while combination delivery had the lowest (85.4%) (Figure 1(a)).Retention through session 22 was highest among in-person participants (22.8%) and lowest among combination participants (13.7%) (Figure 1(a)).Retention measured by the average number of weeks in the program was highest for in-person participants (28.1), followed by distance learning      1).Retention through 44 weeks was highest among in-person participants (32.7%), followed by combination (18.4%), distance learning (17.2%), and online (14.6%) (Figure 1(b)).Participants enrolled in in-person programs were retained to 196.9 days, on average, which was higher than any other delivery mode (Table 1).Those enrolled in combination programs spent the least number of days in the program, on average, 130.5.When examining the average number of days it took participants to reach each session, we observed substantial differences by delivery mode (Figure 2, Table S2).Although the number of days was similar through approximately 8 sessions, online participants were reaching session 12 in 6-16 fewer days and were reaching session 22 in 63-79 fewer days, relative to other delivery modes. 2 shows the outcomes for the study population restricted to those who had at least two sessions with recorded weights and at least one session with recorded weekly physical activity minutes.The distribution of participants by demographic variables was similar to that of the larger study population (Table 1) and is found in Table S1.

Physical Activity. Table
First and last physical activity minutes recorded for participants were displayed regardless of which sessions were their first and last (Table 2).The first-session average weekly minutes was highest for online participants (183.0),followed by distance learning (170.3),combination (132.2), and in person (127.3).The last-session average weekly minutes were highest for distance learning participants (197.4),followed by in person (169.3),online (163.2), and combination (147.2).Only online participants had a lower lastsession average of physical activity minutes than their first-session average, with a difference of -19.8 minutes.In-person participants showed the highest increase in firstto-last-session averages (42 minutes).
Average physical activity minutes increased with the number of sessions completed for all delivery modes except distance learning (Figure 3(a), Table S3).By session 6, participants in all delivery modes were averaging at least

Weight Loss.
For all delivery modes, the average participant percent weight loss gradually increased as the session number increased (Figure 4(a), Table S4).Average percent weight loss among those who attended through session 22 was highest among in-person participants (6.7%) and lowest among combination participants (5.5%), but participants in all delivery modes met the average percent weight loss program goal of 5%.On average, in-person participants met the 5% goal sooner (at session 15) than those in programs using other delivery modes.Participants in programs using combination delivery did not meet the 5% average goal until session 19 (Figure 4(a), Table S4), approximately 70 days later (Figure 2, Table S2).
When comparing average initial weight to average final weight, we saw a decrease across all delivery modes (Figure 4(b), Table S5).The decrease in average weight was greatest for distance learning participants (10.1 lbs.), followed by in person (9.5 lbs.), combination (6.3 lbs.), and online (5.8 lbs.) (Table 2).Overall, participants enrolled in distance learning and in-person organizations represented the highest percentage of those meeting the 5% weight loss goal, 37.2% and 34.3%, respectively (Figure 5).Those enrolled in online and combination organizations represented the highest percentage of participants who gained weight during the lifestyle change program, 22.7% and 17.4%, respectively.Organizations using combination delivery had the highest percentage of participants with only one recorded weight (15.4%), while in-person organizations had the lowest percentage of participants with only one recorded weight (4.8%).
Figure 6 displays cumulative weight loss by delivery mode for all participants.Distance learning organizations had the smallest percentage of participants who lost ≤ 0% of their initial body weight (i.e., gained weight or lost no The value for each session was calculated using weights from participants still in the program at that session who reported a weight at that session.7 Journal of Diabetes Research weight) (14.4%), whereas online organizations had the highest (27.0%).In addition, distance learning organizations had the highest percentage of participants who achieved more than 7% weight loss (23.5%).With respect to meeting the program goal of losing at least 5% of one's body weight, distance learning participants achieved this at the highest rate (36.9%), followed by in person (34.2%), and combination and online (19.2%).Examples are provided to help with interpretation.Results are cumulative (e.g., participants who lost 6% of their weight would also be included in the percent of participants who lost 5% of their weight).The lines do not go to 100% because weight gain was truncated at 1% (i.e., weight loss of -1%).

Discussion
Enrolling and retaining participants in a voluntary, 12month behavioral health intervention is inherently challenging.We found that the median time spent in the National DPP LCP was 6.6 months for in-person participants and a 54.4% of distance learningparticipants lost greater than 3% of their body weight.
8.8% of distance learning participants gained greater than or equal to 0.5% of their body weight.

Participants who gained weight
Participants who lost weight   range of 2.9 to 3.5 months for participants using other delivery modes.Participants often face challenging personal, family, and work constraints that affect retention [4][5][6], but retention is also likely to be influenced by program factors such as the curriculum used, coach skills, program location, and class schedules.A recent survey of in-person participants found that, by far, the biggest factor associated with dropping out was class timing and scheduling [6], suggesting that flexibility of class offerings could be a key way to improve retention.Increased focus on individual participant goals has also been cited as something that might help retain people longer [7].Emotional support and accountability are important to some participants as they move through the program [8].In particular, organizations delivering behavior change interventions using digital platforms such as the Internet or mobile phones may find that lack of personal contact may be reason enough for some participants to drop out [8].Technology-only (asynchronous) driven interventions may allow for easier participant disengagement [9].Retention may be further improved by obtaining more feedback from participants, coaches, and current and prospective delivery organizations to identify and address other relevant factors.
Even though most participants dropped out before attending 22 sessions, results showed that, on average, participants in every delivery mode increased their physical activity (Figure 3(a), Table S3) and lost weight (Figure 4(a), Table S4) as they moved through the program.Despite average session attendance ranging from only 10.7 to 15.0 across the four delivery modes (Table 1), the percentage of participants losing some amount of weight was 76.3% (in person), 63.1% (online), 79.9% (distance learning), and 61.4% (combination) (Figures 5  and 6).Other translation studies have also found that participants lose weight incrementally the more time they spend in a program [10].Furthermore, simply avoiding weight gain is highly protective against diabetes [11].In addition, physical activity interventions have been shown to be independently associated with lowering diabetes risk [12], which contributed to DPRP's addition, in 2021, of an optional outcome measure combining physical activity with a lower weight loss threshold for the National DPP LCP [2].Taken together, these findings suggest that participants will likely derive some health benefits even if they leave the program early, but that the benefits will increase the longer participants stay.
Across all delivery modes, we found that average weight loss reached ≥ 5% by session 19 (~5-7 months) (Figures 2  and 4(a)), and most participants reached ≥ 150 weekly physical activity minutes by approximately session 6 (Figure 3(a), Table S3).However, it is likely that greater time in the program enhances habit formation and longer-term behavioral change.The community guide recommended that programs have a minimum 3-month duration.The guide also concluded that higher intensity programs (e.g., more sessions) lead to greater reduction in new-onset type 2 diabetes, and that programs with longer core phase durations may be more effective [13].In our study, as well as a previous study using DPRP data [14], greater time in the program led to greater average weight loss (with the exception of distance learning) and greater average physical activity minutes (Figures 4(a) and 3(a)).However, our data suggest that, in a real-world setting, retaining all participants to 12 months may not be feasible.
We found that in-person programs outperformed other delivery modes with respect to retention (Figures 1(a) and 1(b)), increases in physical activity (Table 2, Figure 3(b)), and (with the exception of distance learning) weight loss (Tables 2 and S5, Figure 4(b)).Distance learning participants, on average, showed the largest decrease from initial to final weight, 10.1 lbs.and 4.7% (Table 2).In-person and distance learning programs had the highest percent of participants achieving the 5% weight loss goal, 34.3% and 37.2%, respectively (Figure 5).We also noted that inperson participants met the average 5% weight loss goal at an earlier session (15) than other participants, as well as at an earlier time in the program (141 days, or about 4.6 months).Future analyses might examine if participants who achieve weight loss goals early go on to lose additional weight by the end of the program.Factors commonly associated with synchronous delivery (i.e., in-person and distance learning) such as the personal nature of the coach and peer interaction, as well as the sense of community, might contribute to better outcomes.A review of weight loss interventions that use a personal contact component, in addition to mobile phone technology for delivery, was shown to be more effective than those that lacked personal interaction [15].It should be noted that when online delivery of the lifestyle change program includes intensive coach support (other than live delivery), or support through participant groups or online social networks, results that align with the National DPP's programmatic weight loss goal of 5% can be achieved [16].In our study, there may be differences in outcomes due to participant motivation, readiness to attend, and contextual barriers to full engagement.Our results suggest that these are all factors that could improve retention, as well.
Although in-person participants tended to have better program results overall, and outcomes associated with online participants need improvement, online enrollment has surpassed in-person enrollment [3].The expansive reach of online programs can help scale the National DPP LCP, especially as we see that participants who stay in these programs lose weight and increase their physical activity.Furthermore, online and other virtual delivery modes have the potential to reduce barriers to program participation that are more often encountered by people who experience adverse social determinants of health-for example, barriers related to childcare, transportation, distance to the program, and discretionary time.Of course, access to technology can also be a barrier, so serious attempts to reduce health disparities through the National DPP will likely explore how to leverage technology to reach underserved populations.
There are several limitations associated with DPRP data.First, the DPRP collects data on only a limited number of variables to minimize the data collection burden on program delivery organizations.Second, certain data points, such as physical activity minutes, are self-reported by participants.9

Journal of Diabetes Research
The DPRP does not collect information as to how these minutes are measured or reported.In addition, we do not collect information on the specific methods used by virtual organizations to obtain participant weights.Third, the DPRP receives only what organizations report, although it performs quality control checks and audits when necessary.Fourth, the precise nature of combination delivery is often unknown to the DPRP and likely varies by organization (e.g., the percent of sessions attributed to one delivery mode vs. another), precluding strong conclusions about the results for this delivery mode.
Of note, we found the distance learning data challenging to interpret because, even though retention was lower than for in-person delivery and participants' average physical activity minutes per session did not increase over time, overall weight loss was slightly higher than that of in-person participants.This appears to be driven by a high degree of early weight loss, which subsequently drops off over time (Figure 4(a), Table S4).Our findings about distance learning delivery should be considered preliminary because the sample size is only 1%-2% of the in-person or online populations.In future analyses, we plan to examine distance learning in more depth, using data collected from the expanded number of organizations that delivered via this modality during the COVID-19 public health emergency.
Future analysis should focus on outcomes by delivery mode and participant demographics (sex, age, race/ethnicity).Given that certain populations (e.g., men and participants aged 18-44 years) enroll in virtual programs at a higher rate than in-person programs [3], it will be important to examine how this impacts overall delivery mode outcomes.

Conclusions
Since it began in 2012, the Diabetes Prevention Recognition Program has recognized over 4,000 organizations delivering the National DPP lifestyle change program to nearly 700,000 participants, via four different delivery modes.While scaling the program can help reach more of the 96 million Americans with prediabetes [17], we can also focus on participant retention and outcomes.Retention is inherently challenging, but various program changes, such as the curriculum, coach skills, class schedules, and even a change in class location, can improve retention.Participants in in-person programs had the best retention, and participants in in-person and distance learning programs had the highest weight loss, suggesting that the essential features of synchronous delivery might be applied to other delivery modes.Nevertheless, if participants stay in the program long enough, most have positive program outcomes regardless of delivery mode.Importantly, most participants lose weight and increase physical activity if they leave the program early, but their outcomes improve more the longer they stay.These findings highlight the benefits of increased enrollment as well as better retention within the National DPP lifestyle change program.

Figure 1 :
Figure 1: (a) Percent of participants in the National Diabetes Prevention Program (National DPP) lifestyle change program (LCP) retained at each session, by organization delivery mode.(b) Percent of participants in the National DPP LCP retained each week, by organization delivery mode.

Figure 2 :
Figure 2: Average number of days to reach each session attended by participants in the National Diabetes Prevention Program lifestyle change program, by organization delivery mode.

Figure 3 :
Figure 3: (a) Average weekly physical activity (PA) minutes for participants in the National Diabetes Prevention Program (National DPP) lifestyle change program (LCP), by session and organization delivery mode.The value for each session was calculated using PA minutes from participants still in the program at that session.(b) Proportion of participants in the National DPP LCP with first and last entry of weekly PA minutes in each specified interval, by organization delivery mode.Session numbers where the first and last recorded weekly PA minutes were reported varied for individual participants.Of note, the percentage change in those reaching the program goal of 150 minutes per week can be seen in the dark green line.

Figure 4 :
Figure 4: (a) Average weight loss per session for participants in the National Diabetes Prevention Program (National DPP) lifestyle change program (LCP), by organization delivery mode.(b) Average first and last recorded weight for participants in the National DPP LCP, by organization delivery mode.The value for each session was calculated using weights from participants still in the program at that session who reported a weight at that session.
of distance learning participants had only one recorded weight.

Figure 6 :
Figure 6: Cumulative distribution of weight loss outcomes for all participants in the National Diabetes Prevention Program lifestyle change program, by organization delivery mode.

Figure 5 :
Figure 5: Weight loss outcomes for all participants in the National Diabetes Prevention Program lifestyle change program, by organization delivery mode.

Table 1 :
Characteristics of participants who enrolled in the National Diabetes Prevention Program lifestyle change program by December 2018, by organization delivery mode.

Table 2 :
Characteristics of participants who enrolled in the National Diabetes Prevention Program lifestyle change program by December 2018 and who had at least two sessions with recorded weights and at least one session with recorded weekly physical activity minutes, by organization delivery mode.